GitLab CI/CD Catalog

Overview

DevAssure O2 is an AI-powered end-to-end (E2E) testing agent that integrates directly into GitLab CI/CD pipelines. It reads merge request diffs, automatically generates targeted UI tests, executes them on real browsers, and surfaces results in the pipeline and MR — with no test scripts to write or maintain.

Core workflow: reads code diff → maps blast radius → generates tests → executes → reports

The component works on merge request pipelines, branch pipelines, and other GitLab CI jobs. It uses the DevAssure CLI — @devassure/cli.

---

How It Works

When a merge request pipeline runs:

1. The DevAssure O2 agent reads the code diff from the MR or branch.
2. It identifies the impacted areas (blast radius) of the changes.
3. It generates comprehensive end-to-end UI tests in natural language to validate those changes.
4. It executes the generated tests in real browsers.
5. It surfaces bugs in the pipeline and in the MR, with reports and GitLab Tests tab integration via JUnit XML.

---

Prerequisites

• Runners: Unix-like GitLab runners only — Linux container-based or macOS shell runners. Windows runners are not supported.
• Git: The test command expects a normal git checkout of the repository. For merge requests, ensure enough history is available for diffs and branches to resolve. Set GIT_DEPTH: 0 in the job if needed.
• Node.js: Version 20 or above. The component validates this automatically.
• DevAssure API Token: Required for authentication. See Authentication below.

---

Authentication

Creating a DevAssure API Token

1. Log in at https://app.devassure.io or sign up for free at https://app.devassure.io/sign_up.
2. Navigate to Account → Settings → Token.
3. Generate a new API token.
4. Save it as a masked (and protected, if using protected branches) GitLab CI/CD variable named DEVASSURE_TOKEN at the project or group level.

Configuring the Token

The component resolves the token in order of precedence:

1. token component input (explicit pass-through)
2. DEVASSURE_TOKEN CI/CD variable (recommended)

Recommended approach — store the token as a masked variable in Settings → CI/CD → Variables and leave the token input empty. The component reads it automatically.

---

Component Location

The CI component is exposed from this repository at:

templates/devassure/template.yml   (component name: devassure)

---

Installation & Basic Setup

Include the component in your .gitlab-ci.yml using a pinned tag for stability:

include:
  - component: $CI_SERVER_FQDN/devassure/devassure/devassure@1

Set DEVASSURE_TOKEN as a masked/protected CI/CD variable in your project or group.

Security note: Always pin to a specific tag (e.g., @1 or @1.0.0). Avoid @latest in component references to ensure pipeline stability.

---

Commands

The component supports the following commands via the command input. The default command is test.

Command	Description
setup	Installs and configures the DevAssure CLI
test	Generates and runs E2E tests from an MR or branch diff
run	Executes existing tests with optional filters
summary	Retrieves a test session summary
archive / archive-report	Archives the test report for a session

test — Generate and Run Tests from a Diff

include:
  - component: $CI_SERVER_FQDN/devassure/devassure/devassure@1
    inputs:
      command: test
      minimum_score: 80
      workers: 2
      path: e2e/tests
      source: $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME
      target: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
      commit_id: $CI_COMMIT_SHA
      url: https://example.com
      environment: staging
      headless: "true"

run — Execute Existing Tests with Filters

include:
  - component: $CI_SERVER_FQDN/devassure/devassure/devassure@1
    inputs:
      command: run
      workers: 2
      path: e2e
      filter: smoke
      query: "login flow"
      tag: nightly
      priority: high
      folder: reports
      url: https://example.com
      environment: staging
      headless: "false"

summary — Retrieve Session Summary

include:
  - component: $CI_SERVER_FQDN/devassure/devassure/devassure@1
    inputs:
      command: summary
      session_id: sess_123

If session_id is not provided, the component runs devassure summary --last.

archive — Archive a Test Report

include:
  - component: $CI_SERVER_FQDN/devassure/devassure/devassure@1
    inputs:
      command: archive
      session_id: sess_123

If session_id is not provided, the component runs devassure archive-report --last.

setup — Explicit Token Pass-Through

include:
  - component: $CI_SERVER_FQDN/devassure/devassure/devassure@1
    inputs:
      token: $DEVASSURE_TOKEN
      command: setup

---

Inputs Reference

All inputs are optional.

Input	Default	Description
stage	test	Pipeline stage where the generated DevAssure job runs
image	node:24-bookworm	Container image used to execute the job
command	test	DevAssure command: setup, test, run, summary, archive, or archive-report
token	(empty)	DevAssure API token. Falls back to DEVASSURE_TOKEN CI/CD variable if empty
path	(empty)	Relative path to run from (useful when tests live in a subdirectory)
source	(empty)	Source branch for test command scope and branch checkout
target	(empty)	Target branch for test command baseline comparison
commit_id	(empty)	Commit SHA for test command scope
filter	(empty)	Filter expression for narrowing run command execution
query	(empty)	Query string for selecting tests in run
tag	(empty)	Tag-based selector for run
priority	(empty)	Priority-based selector for run
folder	(empty)	Folder selector for run
url	(empty)	Application URL under test (used by test and run)
headless	true	Headless browser mode for test and run ("true" or "false")
session_id	(empty)	Session ID for summary or archive (defaults to latest session)
archive	true	Set to false to skip archive-report --last and artifact archiving for test/run
minimum_score	75	Minimum passing score threshold for test/run. Set to 0 to disable
workers	0	Parallel worker count. Use 0 for CLI default; must be an integer > 0 when set
environment	(empty)	Environment name passed to test/run (e.g., staging, qa, production)
verbose	(empty)	Enable verbose logging (true / false)
debug	(empty)	Enable debug logging (true / false)

---

Command Parameter Mapping

The component forwards supported inputs to the CLI using --<arg-name>="<arg-value>" format.

Command	Forwarded Parameters
setup	(none) — optional verbose / debug flags
test	path, source, target, commit_id, url, workers, environment, headless
run	path, filter, query, tag, priority, folder, url, workers, environment, headless
summary	session_id (when set), otherwise --last
archive / archive-report	session_id (when set), otherwise --last

Note: headless always defaults to true and is always passed for test and run commands. session_id and --last are mutually exclusive — the component ensures only one is passed.

---

Score Validation

For test and run commands, the component automatically runs devassure summary --last after execution and validates the score against minimum_score.

Score validation behavior:

• If minimum_score is non-numeric or <= 0 → validation is skipped.
• If the score is missing or N/A → the job fails.
• If the score is below minimum_score → the job fails.
• If the score meets or exceeds minimum_score → the job passes.

To disable score validation, set minimum_score: 0.

---

Artifacts & Reports

Job Artifacts

When archive is not false and the command is test or run, archived reports are written under:

.devassure-artifacts/

These are exposed as job artifacts with expire_in: 7 days and can be downloaded from Build → Job artifacts.

GitLab Tests Tab Integration

For test and run commands, JUnit XML is published as artifacts:reports:junit, making test results visible directly in the GitLab Tests tab of the pipeline.

Viewing Reports

Downloaded artifacts (.zip) can be opened using:

• DevAssure CLI (@devassure/cli)
• DevAssure VSCode Extension

---

Runner Sizing Recommendations

Workers / Concurrency	Recommended Spec
Baseline (default)	4 vCPU / 16 GB RAM
Higher concurrency	Scale vCPU and RAM proportionally to avoid CPU and memory contention

Increasing workers without scaling the runner will cause performance degradation and potential test failures due to resource contention.

---

Publishing to GitLab Catalog

Recommended release flow for component maintainers:

1. Merge changes to the default branch.
2. Create a semantic version tag (e.g., 1 or 1.0.0).
3. Let .gitlab-ci.yml run release:catalog on tags.
4. Consumers pin the component include with @<tag>.

---

Security & Best Practices

• Store DEVASSURE_TOKEN as a masked and protected CI/CD variable — never hardcode it in .gitlab-ci.yml.
• Pin component references to specific tags (e.g., @1). Avoid @latest.
• Keep runner resources aligned with workers count (see Runner Sizing Recommendations).

---

Local Execution

The DevAssure O2 agent can also be invoked locally using the DevAssure CLI npm package:

npm install -g @devassure/cli
devassure test --url https://your-app.com

See the DevAssure CLI documentation for full local usage details.

---

Troubleshooting

GLIBC_2.35 not found

Cause: The runner image is too old for the packaged DevAssure agent runtime.

Fix: Use a newer Linux base image such as node:24-bookworm, ubuntu:22.04, or newer. Override the component image with inputs.image if your organization provides a compatible custom base.

---

FAQ

How does minimum_score work?

minimum_score is the lowest test score that still passes the job. The default is 75. If the score falls below this value, the job fails; if it meets or exceeds the threshold, the job passes. Set minimum_score: 0 to disable score validation entirely.

Does DevAssure charge per test run?

No. The O2 agent is $25 per agent per month with unlimited executions — no token-based metering and no per-run or per-call charges. Add agents if you need more parallelism. Current plans are on the pricing page.

How do I view the complete test report?

Download the job artifacts from Build → Job artifacts and open them with the DevAssure CLI or the DevAssure VSCode extension.

Can I run the agent locally?

Yes. Install @devassure/cli via npm and run devassure commands directly from your local machine.

Can I use session_id with the summary command?

Yes. Pass a specific session_id to retrieve the summary for that session. If omitted, the component defaults to devassure summary --last (most recent session). The --last flag and session_id are mutually exclusive.

---

Related Resources

• DevAssure Web Portal
• DevAssure CLI npm Package
• Sign Up for Free
• Pricing